Skip to content

The corpus becomes two tiers, and the contract tier becomes chosen names - #482

Merged
derek73 merged 16 commits into
masterfrom
claude/corpus-names-strategy-cadcf7
Sep 1, 2026
Merged

The corpus becomes two tiers, and the contract tier becomes chosen names#482
derek73 merged 16 commits into
masterfrom
claude/corpus-names-strategy-cadcf7

Conversation

@derek73

@derek73 derek73 commented Sep 1, 2026

Copy link
Copy Markdown
Owner

The differential corpus becomes two tiers, and the contract tier becomes something someone chose. Closes #468; lands the executable-invariant and documentation halves of #469 (its CJK third-shape question stays open).

What changes

Tier split (#468). corpus.jsonl (scraped from v1's test banks) and corpus_issues.jsonl (harvested from the tracker) become radar: their diffs still classify against the ledger so release notes keep their grouping, but an unmatched radar diff prints under UNCLASSIFIED (radar) and cannot fail the run or demand a ledger rule. corpus_cjk.jsonl, corpus_rules.jsonl and the new corpus_shapes.jsonl are contract and keep the full discipline. The roster is fail-closed like the floors. Nothing is deleted: promotion is a tests/v2/cases.py row plus a shape= tag — a name enters the contract by being chosen, which is #468's "recorded once when the name is admitted". A [[never]] exclusion outranks the tier (an excluded shape was chosen too; the 1.4.0 ledger's own why fields promise exit-1 on names that live only in radar files).

Recovered v1 labels. corpus.jsonl regenerated at the same pinned ref (2d5d8c2) carrying the test-method names the original scrape discarded (test_title_with_conjunction and kin); name set proven byte-identical (486), so every _CORPUS_CLAIMS digest stood. Radar triage now reads what each string exercised in v1.

Shape-matrix contract corpus (#469's inventory as the index). tools/differential/shapes.py records each input shape's notation, the name_order it is an input shape for, and the oldest baseline whose worker can honor that order. Shape-tagged case rows generate corpus_shapes.jsonl (the corpus_cjk pattern with an explicit-tag predicate, pinned by the same guard shape); Case.__post_init__ rejects a tag whose shape disagrees with the row's declared order, a locale-carrying row, or CJK text. Coverage is answerable: build_shapes_corpus.py --coverage prints names-per-shape (today 1/4/1/4/3 for shapes 1–5).

Order-aware gate. The worker protocol moves to {"name", "order"} entries; a family-first entry is compared under its shape's order on both sides, on the v2 surface alone, from baseline 2.0.0 up — and skipped, with a printed per-file count, below it. "We don't care what this name does in other orders" is structural, not a ledger exception. Ledger rules gained an optional orders narrowing key (legal set borrowed from the shape inventory), because an order-scoped rule that cannot be scoped would absorb a default-order regression of the very fold it describes — the pinned test holds that default-order diff UNEXPLAINED.

#469 docs and the executable invariant. usage.rst documents forms 4 and 5; customize.rst carries the caller-facing correspondence (form 4 is the family-comma format written inline) with the corrected, dated measurement — 0/630 → 603/630 (2026-08-30), not the sliced-vocabulary 450/450 in #469's own text — and its three limits. tests/v2/test_order_correspondence.py generates 96 pairs and sweeps the whole shipped particle vocabulary as a negative control, asserting the disagreeing set is exactly {vd, mc, do}.

Design record. A new decisions.md section ("differential-ledger, the corpus-tier arc") carries the decisions; mechanics live in tools/differential/README.md.

Verification

  • Full suite 6334 passed; mypy clean (110 files); ruff clean; sphinx doctests 244/0.
  • Gate exits 0 at all four baselines: intentional diffs 226 / 205 / 113 / 0 at 1.4.0 / 2.0.0 / 2.1.0 / 2.2.0, unexplained 0 and radar-unclassified 0 everywhere; 1113 comparisons (13 shape entries, 4 deduping into existing default-order names), 7 family-first entries skipped at 1.4.0.
  • The 2.x arrivals are classified under literal-anchored, order-scoped rules carrying the case rows' own feat(#395)/fix(#399)/fix(#296) classifications; _CORPUS_CLAIMS re-recorded with every grown claim traced to a shapes-corpus arrival.

Follow-ons (not in this PR)

Authoring net-new variation rows shape by shape from the coverage report; #469's CJK third-shape question; a possible later corpus_cjk fold into the shapes corpus; a correction comment on #469's retracted numbers.

🤖 Generated with Claude Code

derek73 and others added 10 commits September 1, 2026 03:47
Chosen names (rules, cjk) keep the full discipline: an unmatched diff
is UNEXPLAINED and the run exits 1. Scraped and harvested names
(corpus.jsonl, corpus_issues.jsonl) become radar: their diffs still
classify for release-note grouping, but an unmatched one prints under
UNCLASSIFIED (radar) and cannot fail the run or demand a ledger rule
(#468). The roster is fail-closed like _CORPUS_FLOORS. Corpus lines
may now be bare strings or {"name": ...} objects, ahead of the
label-bearing regeneration.

A [[never]] exclusion is fatal on both tiers, not just contract:
classify() returns None both for "no rule matched" and for "an
exclusion refused this name", and only the first belongs to the tier
split. An exclusion was chosen -- someone wrote its `why` and its
`examples` -- so it stays UNEXPLAINED wherever the name it refuses
happens to sit.

Measured at all four baselines: every classified count identical to
the pre-change reference, radar unclassified 0.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…aped from

The AST walk kept each string literal and discarded the method name
around it -- test_title_with_conjunction and kin, which said what
shape each name exercised. Regenerated at the same pinned ref
(2d5d8c2) with the labels kept: the name set is byte-identical (486,
verified by set comparison), so every _CORPUS_CLAIMS digest stands.
The radar report now shows a moved name's v1 labels, which is the
triage #468 asked for on names nobody chose. A HumanName(...) call is
labelled by its nearest enclosing function, falling back to the
source filename for one at module scope, so a future regeneration at
a still-earlier ref cannot silently drop a module-level call the way
a function-scoped-only walk would.

Also routed two other bare-string-assuming readers through the same
format-tolerant extraction: conftest.py's differential_corpus() (used
by test_locales.py and test_properties.py) and the manual heredoc
recipes in test_ledger_guards.py and expected_since_1.4.0.toml that
recompute corpus-derived counts by hand. corpus_cjk.jsonl stays
bare-string on purpose (it is generated, not scraped) and its own
reader is untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
shapes.py records each input shape's notation, the name_order it is
an input shape for, and the oldest baseline whose worker can honor
that order (2.0.0 -- Policy and both family-first constants shipped
there). The worker protocol moves from bare name strings to
{name, order} objects; an order-bearing entry is compared on the v2
surface alone, under its order on both sides, and is skipped -- with
a printed count -- at baselines below its shape's minimum. Inert in
this commit: no corpus carries a shape yet, and all four baselines
measure identical to the pre-change reference.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Case.shape names an entry in tools/differential/shapes.py; tagging a
row is the act of admission to the differential's contract corpus
(#468's 'recorded once when the name is admitted'). __post_init__
rejects a tag whose shape implies a name_order the row's policy does
not declare, and a ledger guard holds the id set equal to the
inventory. Tags the given-first exemplars and every family-first
name_order row.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The tier and order-protocol tests landed with comma-joined imports
(E401) and unannotated _fake signatures (ANN001/ANN202), which the
repo-wide ruff gate rejects. Mechanical: no test logic moves.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tagged case rows

build_shapes_corpus.py projects every shape-tagged cases.py row into
corpus_shapes.jsonl -- the corpus_cjk pattern with an explicit-tag
predicate -- pinned by the same guard shape. Family-first entries are
compared under their shape's order from baseline 2.0.0 up and skipped
below it. Classifies the new family-first comparisons at 2.0.0/2.1.0
under feat(#395) and fix(#399), with literal-anchored rules.
_CORPUS_CLAIMS re-recorded; every grown claim verified to reach only
shapes-corpus arrivals.

A rule may now carry an optional `orders` list, and the four new rules
do. A name compared twice can move the SAME roles for opposite
reasons: 'de la Cruz Juan Carlos' diffs {family, given, middle} under
a declared family-first order because #395's fold is intended there,
and would diff exactly those three roles under the DEFAULT order if
that fold ever leaked where rules.md#P1 forbids it. An order-blind
rule absorbs the leak and calls it intentional -- #372's failure mode
aimed at the most plausible regression of the change the rule
describes. classify() and dormant_rules() now ask the same
order-aware question; exclusions stay order-blind, refusal being
monotone and so loud rather than silent.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Form 4 under FAMILY_FIRST must parse as its form-2 comma twin, title
movement included. Generated over a small forked vocabulary (never-
given/ambiguous/multi-token particles, an ASCII and non-ASCII
family, doubled middles) rather than sampled, and vocabulary-blind on
purpose: decisions.md#P6 (the 2026-08-30 #467 entry) measured 0 of
630 -> 603 of 630 over the whole 70-particle vocabulary, with the 27
failures being exactly the particle-and-suffix trio (vd, mc, do) -- a
real limit, not noise (rules.md#P6) -- and #466, the rejected
predecessor, lost a given name outright on "van Berg Jan de" and
promoted a post-nominal into the given slot on "Berg Jan Jr. de".
decisions.md#two-input-invariants recomputes the same correspondence
a second way (a 6x6 family/given spread per particle) and supplies
this file's negative-control recipe. An earlier draft of this file
cited "0/216 -> 216/216", which decisions.md#P6 explicitly retracts
as a sliced-vocabulary artifact (`sorted(particles)[:14]`, which
happened to exclude the trio); this file's own count -- 96
parametrized pairs plus a full-vocabulary negative-control sweep --
is kept separate from and never confused with either retracted or
surviving number.

The comparison now also covers the ambiguity-kind set alongside the
seven role fields (the compare.py comparator shape), since
decisions.md#P6 (#405) records that this design's own repaired
failure was exactly an ambiguity asymmetry between two writings of
one name; all 96 pairs agree on both. A second, non-parametrized test
sweeps the entire shipped particle vocabulary in the minimal pair and
asserts the disagreeing set is exactly {do, mc, vd} -- a recorded
literal per mechanisms.md#RECORDED-ROSTERS, guarded by a
mechanisms.md#VOCABULARY-OVERLAP-AS-PRECONDITION precondition that
the fixture family/given is not itself particle vocabulary -- and
resolves decisions.md#two-input-invariants' "Open: #469" marker for
an executable form.

Form 5 is absent by construction -- no comma format puts the given
name last (#469).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…pondence

usage.rst gains forms 4 and 5 beside the three given-first
arrangements, with the particle slot's reason in one sentence.
customize.rst carries the caller-facing correspondence -- form 4 is
the family-comma format written inline, 603 of 630 measured
2026-08-30 -- with its three limits, the particle-and-suffix trio
among them (the earlier unqualified statement predates the
sliced-vocabulary retraction in decisions.md#P6). rules.md gets a
pointer note only, worded to admit no example into the generated
corpus (verified against the rules-corpus pin). Closes the
documentation half of #469.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…corpus

Two comments written by the labels commit said the other three
corpora stay bare strings; the shapes corpus falsified them one
commit later. The remaining arity claims either name all five,
verified, or stop counting.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The tier split, the promotion path, exclusions outranking the tier,
the orders narrowing key, and shape-carries-order each get their
dated entry beside the earlier differential-ledger arcs; mechanisms'
narrowing model and roster inventory catch up. Mechanics stay in
tools/differential/README.md, which these entries cite.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@derek73 derek73 added this to the v2.3 milestone Sep 1, 2026
@derek73 derek73 added docs Documentation fixes and updates tooling tests labels Sep 1, 2026
@derek73 derek73 self-assigned this Sep 1, 2026
@codecov

codecov Bot commented Sep 1, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.66%. Comparing base (f18a42a) to head (92b3b96).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #482   +/-   ##
=======================================
  Coverage   98.66%   98.66%           
=======================================
  Files          45       45           
  Lines        3218     3218           
=======================================
  Hits         3175     3175           
  Misses         43       43           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

derek73 and others added 6 commits September 1, 2026 10:13
_load_entries ignored every key it did not know, so a corpus line
reading {"name": ..., "shpae": 4} declared no shape at all: the name
was compared under the default order and the run reported it as a
full pass. The narrowing the author wrote was dropped silently, which
is the failure mode validate_rules already rejects on the ledger side.

The three computed keys are rejected rather than obeyed. main()
writes "order", "tier" and "file" onto every entry, so a line
carrying one had it overwritten -- and "order" is the key the WIRE
protocol documents, which makes it the one a corpus author reaches
for when they want a family-first comparison.

The docstring's rationale for the `shape` type check was wrong: the
resolution loop runs before the worker, so a bad id is not a late
crash. `true` resolves against shapes.py's entry 1 by hash equality
and the line is compared under that shape's order, silently.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…lind absorption is visible

`orders` closed one leak and left its mirror open. A rule with no
`orders` key claims a diff under EVERY order, so a legacy rule whose
regex happens to reach an order-bearing name absorbs that name's
family-first regressions too -- and legacy rules sort ahead of the
scoped ones written for the same string. Measured: the order-blind
`fix(#399) a maiden marker bounds the particle chain that swallowed
it` rule reaches `de la Cruz née Vega`, which corpus_shapes.jsonl
compares under both family-first orders, so a family-first-only
{family, maiden} regression there would have classified as intended.

A rule could not say "default order only": `orders` members are
constant NAMES, the default order is the absence of one, and TOML has
no null inside an array. `_legal_orders()` now returns the
shapes-derived set plus the "DEFAULT" sentinel, and _entry_matches
reads a comparison order of None as that member.

The two legacy rules are scoped in both 2.x ledgers, to the scopes
their diffs actually come from: fix(#399) to ["DEFAULT"], and
fix(#296) `John Smith, Dr.` to ["DEFAULT", "FAMILY_FIRST"], which
deliberately explains the shape-4 family-first diff too.

Order-blind rules stay legal, so the absorption is made visible
rather than illegal: a run prints an ORDER-BLIND block naming each
(issue, name, order) where a rule with no `orders` key explained an
order-bearing diff. Informational, outside the exit code, and empty
at all four baselines today.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
_CORPUS_CLAIMS recorded a rule's regex reach, its narrowing roles and
a digest of the names it claims -- and deleting `orders` from a
shipped rule moves none of the three. The rule goes back to claiming
every order, which is the widening the key exists to prevent, and the
roster that exists to refuse quiet widenings had nothing to say.

_Claim gains a fourth entry: the rule's `orders` sorted, or None when
it carries no key at all. No default, so a new entry has to state
which one it is. Every roster entry is re-recorded; the twelve scoped
rules across the two 2.x ledgers carry their tuples.

Verified by removing one `orders` key in the working tree: the roster
test fails with the recorded tuple against a computed None.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…order branch, and order-aware dormancy

Four gaps, each behind a mutation that left the suite green.

The (name, order) dedup key had no test: every order test above used
a single-shape corpus, where the key and the bare name agree.
Regressing it to the name alone dropped one of two comparisons and
printed a corpus count one smaller.

The order-scoped rules on the shipped ledgers were pinned nowhere.
The sweep asks every one of them, over the names its own regex
reaches, that it declines the default-order reading and claims
something under an order it lists -- with a vacuity guard, since the
sweep is silent on a ledger that happens to carry no scoped rule.

The worker's order branch was compiled by one test and fed by
another, and executed by neither. It is exec'd in-process here, where
the template's `import nameparser` resolves to the checkout, so its
emitted row is compared against the tree's own family-first parse --
the equality every order-bearing comparison rests on.

And an order-scoped rule whose behavior reverted was diagnosed
"reverted" by nothing: read order-blind it matches the default-order
diff of the same name, sees another rule win it, and reports
"shadowed" -- sending someone to delete a rule that is not redundant.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…S.md's radar clause

Four recompute recipes in decisions.md said "the four
tools/differential/corpus*.jsonl files" over five corpora; the glob
already says how many, as the same recipe's copy in
tests/test_capitalization.py already had it.

The trio explanation in customize.rst and in
tests/v2/test_order_correspondence.py had the causality inverted.
Measured: parse("Ménil, Christophe vd") reads family 'vd Ménil' --
the particle attachment winning on the comma side, which is the
scope rules.md#P6 states it in -- while the FAMILY_FIRST parse of
"Ménil Christophe vd" reads family 'Ménil' and suffix 'vd'. It is
that asymmetry that breaks the correspondence, not a precedence
holding on both sides; as written a reader could conclude
family-first reads 'vd Ménil' too.

The correspondence module's standing "96 pairs" is now the product of
the tuples that decide it, and its one file:line citation names the
test instead.

AGENTS.md: exit 0 stopped meaning "every diff is classified" when the
tier split landed. Release step 1 now says to read the UNCLASSIFIED
(radar) block, and the Release-log claims line says what the
classified summary does and does not cover.

Also: the radar rows print Role-named field lines, so the legend
above them is no longer gated on `unexplained` alone; the
corpus_rules.jsonl floor comment says 252 rather than 241; and two
comments drop historical clauses nothing can check.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 1ca20a6 rename changed the input string everywhere but left the
expected given name behind in three places -- the shipped 2.2.0
release-log bullet and two spots in decisions.md#P1 -- exactly the
half-updated-example failure that commit's own message warns about.
Measured: Parser(policy=Policy(name_order=FAMILY_FIRST)) reads
'de Mesnil Jean' as family 'de Mesnil', given 'Jean'. The
self-consistent Juan inputs ('de Mesnil, Juan' in the user docs and
test rows, 'van Mesnil Juan' in P1) are deliberate inputs, not
errors, and stand.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation fixes and updates tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Corpus names become permanent contract without recording what shape each one exercises

1 participant